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(54) Method for automatic text placement in digital images 

(57) A metn ^ d e( J°/ SlS?St@ a human understandable 
item, such as a clption,°text or a figurative element, in a 
digital image includes the steps of (a) digitally process- 
ing the digital image to recognize and identify an optimal 
location in the digital image for placing the item and (b) 
modifying the placement of the item in relation to human 
understandable image content so as to minimally 
obscure such human understandable image content in 
the digital image. The recognition and identification of 
an optimal location further includes the identification of 
an optimal open space region and then finding an opti- 
mal location for the item within the open space region. 
The placement of the item may be done so as to mini- 
mally obscure human understandable image content 
either in other locations of the optimal open space 
region, or in regions of the digital image adjoining the 
optimal open space region. Placement of the image 
may further be based on evaluating the proportion of 
image pixels in the item that occlude pixels outside of 
either the optimal location or the optimal region. Moreo- 
ver, one or more attributes of the item, such as the color, 
size or shape of the item, may be modified in relation to 
the human understandable image content in the digital 
image to enhance the prominence of the item in the dig- 
ital image. 
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Description 

FIELD OF THE INVENTION 

[0001] The invention relates generally to the field of 
digital image processing and digital image understand- 
ing and, more particularly, to the automatic placement 
of an item such as a figurative element or text within an 
image. 

BACKGROUND OF THE INVENTION 

[0002] The creation of custom picture postcards or 
other picture souvenir items often presents a customer 
with an opportunity to place a text caption, a set of 
thought bubbles, or a figurative element within a digital 
image. Currently many systems offering these services 
such as digital print kiosks or web sites offer a user two 
options, a fixed location on the image (without regard to 
image content) or a location chosen by the user. Often 
the choice of the item's size or color is similarly limited. 
Conversely, the choices may be essentially limitless if 
the system offers little or nor guidance to the user. For a 
great number of people, this can lead to time consuming 
indecision. Limiting the options of the user may speed 
up the process, but will frustrate the user if the options 
offered are not aesthetically pleasing. 
[0003] U.S. Patent No. 5,901,245, which issued 
May 4, 1999 to the assignee of this application, 
describes a method for detecting open space in an 
image. The method involves the creation of an activity 
map for determining the activity of the digital image, and 
then determining which pixels in the activity map are 
below a threshold. Open space is identified as contigu- 
ous pixels of the activity map having values below the 
determined threshold. This method identifies regions 
within the image which are suitable for the inclusion of 
text or figurative elements into the image. While the 
location of open space does assist the user in finding an 
appropriate location in an image for an item, the mere 
location of an open space region does not ineluctably 
lead to an aesthetically pleasing use of the space. 
[0004] Consequently, a need exists to more fully 
automate the process of placing a desired item within 
an open space in a given digital image, particularly to 
render the item and, if necessary, its surroundings to 
produce an aesthetically pleasing final image. 

SUMMARY OF THE INVENTION 

[0005] The present invention is directed to over- 
coming one or more of the problems set forth above. 
Briefly summarized, according to one aspect of the 
present invention, the invention resides in a method for 
placing a human understandable item in a digital image 
including the steps of (a) digitally processing the digital 
image to recognize and identify an optimal location in 
the digital image for placing the item and (b) modifying 



the placement of the item in relation to human under- 
standable image content so as to minimally obscure 
such human understandable image content in the digital 
image. 

5 [0006] The recognition and identification of an opti- 
mal location further includes the identification of an opti- 
mal region comprising open space in the digital image 
and then finding an optimal location within the optimal 
region. The placement of the item may be done so as to 
w minimally obscure human understandable image con- 
tent in other locations of the optimal region, or in regions 
of the digital image adjoining the optimal region. The 
modified placement of the image may be based on eval- 
uating the proportion of image pixels in the item that 
15 occlude pixels outside of either the optimal location or 
the optimal region. Typical items processed by this 
method include a caption, text or a figurative element. 
[0007] Alternately, the invention may be seen to 
reside in a method for placing a human understandable 
20 item in a digital image so as enhance the prominence of 
the item in relation to human understandable content in 
the digital image, including the steps of (a) digitally 
processing the digital image to recognize and identify 
an optimal location in the digital image for placing the 
25 item and (b) modifying one or more attributes of the item 
in relation to the human understandable image content 
to enhance the prominence of the item in the digital 
image. Such attributes include the color, size or shape 
of the item. 

30 [0008] The present invention has the advantage of 
automatically choosing a set of locations, sizes, and 
colors for a desired item to be placed within a given dig- 
ital picture that are aesthetically pleasing, and thereby 
curtailing the time and effort required by the user for 

35 said task. 

[0009] These and other aspects, objects, features 
and advantages of the present invention will be more 
clearly understood and appreciated from a review of the 
following detailed description of the preferred embodi- 

40 ments and appended claims, and by reference to the 
accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

45 [0010] 

FIG. 1 is a diagram of the steps involved in auto- 
matically placing, sizing, and coloring text or figura- 
tive items within a digital image in accordance with 

so the invention. 

FIG. 2 is a diagram of a method for determining sig- 
nificantly sized contiguous regions in an image that 
do not contain the main subject, and which may 
therefore be used as candidate regions for locating 

55 items in accordance with the invention. 

FIGS. 3A and 3B are diagrams of a method for 
determining an optimal candidate region for locat- 
ing items according to the invention. 
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FIG. 4 is a perspective diagram of a computer sys- 
tem for implementing the present invention. 

DETAILED DESCRIPTION OF THE INVENTION 

[0011] In the following description, a preferred 
embodiment of the present invention will be described 
as a software program. Those skilled in the art will read- 
ily recognize that the equivalent of such software may 
also be constructed in hardware. Because image 
manipulation algorithms and systems are well known, 
the present description will be directed in particular to 
algorithms and systems forming part of, or cooperating 
more directly with, the method in accordance with the 
present invention. Other aspects of such algorithms and 
systems, and hardware and/or software for producing 
and otherwise processing the image signals involved 
therewith, not specifically shown or described herein 
may be selected from such systems, algorithms, com- 
ponents and elements known in the art. Given the 
description as set forth in the following specification, all 
software implementation thereof is conventional and 
within the ordinary skill in such arts. 
[0012] Still further, as used herein, the computer 
program may be stored in a computer readable storage 
medium, which may comprise, for example; magnetic 
storage media such as a magnetic disk (such as a 
floppy disk) or magnetic tape; optical storage media 
such as an optical disc, optical tape, or machine reada- 
ble bar code; solid state electronic storage devices such 
as random access memory (RAM), or read only mem- 
ory (ROM); or any other physical device or medium 
employed to store a computer program. Before describ- 
ing the present invention, it facilitates understanding to 
note that the present invention is preferably utilized on 
any well-known computer system, such a personal 

computer. 

[0013] Referring initially to( fig. 4J there is illustrated 
a computer system 110 for implementing the present 
invention. Although the computer system 110 is shown 
for the purpose of illustrating a preferred embodiment, 
the present invention is not limited to the computer sys- 
tem 110 shown, but may be used on any electronic 
processing system. The computer system 1 10 includes 
a microprocessor-based unit 112 for receiving and 
processing software programs and for performing other 
processing functions. A display 114 is electrically con- 
nected to the microprocessor- based unit 112 for dis- 
playing user-related information associated with the 
software. A keyboard 1 1 6 is also connected to the 
microprocessor based unit 1 12 for permitting a user to 
input information to the software. As an alternative to 
using the keyboard 1 16 for input, a mouse 118 may be 
used for moving a selector 120 on the display 114 and 
for selecting an item on which the selector 120 overlays, 
as is well known in the art. 

[0014] A compact disk-read only memory (CD- 
ROM) 122 is connected to the microprocessor based 



unit 1 12 for receiving software programs and for provid- 
ing a means of inputting software programs and other 
information to the microprocessor based unit 1 12 via a 
compact disk 124, which typically includes a software 
5 program. In addition, a floppy disk 126 may also include 
a software program, and is inserted into the microproc- 
essor-based unit 1 1 2 for inputting the software program. 
Still further, the microprocessor-based unit 112 may be 
programmed, as is well known in the art, for storing a 
10 software program internally. The microprocessor-based 
unit 1 12 may also have a network connection 127, such 
as a telephone line, to an external network, such as a 
local area network or the Internet. A printer 128 is con- 
nected to the microprocessor-based unit 112 for printing 
75 a hardcopy of the output of the computer system 1 1 0. 
[0015] Images may also be displayed on the display 
114 via a personal computer card (PC card) 130, such 
as, as it was formerly known, a PCMCIA card (based on 
the specifications of the Personal Computer Memory 

20 Card International Association) which contains digitized 
images electronically embodied in the card 130rThe PC 
card 130 is ultimately inserted into the microprocessor 
based unit 1 12 for permitting visual display of the image 
on the display 114. Images may also be input via the 

25 compact disk 124, the floppy disk 126, or the network 
connection 127. Any images stored in the PC card 130, 
the floppy disk 126 or the computer disk 134, or input 
through the network connection 127, may have been 
obtained from a variety of sources, such as a digital 

30 camera (not shown) that captures an original scene or a 
scanner (not shown) that scans an original material, 
such as a silver halide film. 

[0016] The invention concerns the treatment of 
human understandable items, such as captions, text or 

35 figurative elements, that are suitable for locating within 
an open space in an image. Captions include titles, 
such as "Birthday Party". Text would include descrip- 
tors, such as (continuing the example) the names of 
those at the party, while figurative elements would 

40 include digital icons or caricatures (e.g., birthday can- 
dles), logos, thought bubbles (with text inside), and the 
like. Such human understandable captions, text, and 
figurative elements will be referred to together from time 
to time in this description and the claims as an n item(s)\ 

45 An "open space", as the term will generally be used 
herein, is defined as a contiguous region of an image 
having smoothly varying color (for a color image) or a 
smoothly varying density (for a monochrome image). 
Some examples of this are clear blue skies, sandy 

50 beaches, or interior walls and ceilings. Mildly textured 
regions may also be considered to be open spaces 
such as grassy fields, water, etc. These open space 
regions often present optimal locations for items that 
obscure the least image content or information. 

55 [0017] There are several activities involved in the 
process of automatically placing such human under- 
standable items within regions of images. The item is 
rendered into a pixel map. Areas of the image suitable 
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for item placement are identified. A suitability function is 
applied to the identified regions, which provides some 
indication of the relative value of placing the item in a 
given location. Of particular concern is the extent to 
which human understandable image content in the dig- 5 
ital image may be obscured by the placement of the 
item. A color is selected for the item, and then the item 
is rendered into the image. These activities can be 
reduced to several stages that are each important in the 
effective automatic placement of a desired item in a to 
given digital image: Referring to Figure 1, after an 
image and an item to be placed in the image are input to 
the computer 1 10 in step 10, an optimal region and an 
optimal location within the region for the placement of 
an item are established in stage 20. In the second stage 15 
30, the size and/or shape of the item is adapted for opti- 
mal placement in the selected optimal location within 
the selected optimal region, particularly so as to mini- 
mally obscure human understandable image content 
elsewhere in the image. (Note that such minimal 20 
obstruction of human understandable content can be 
accomplished by keeping the item to the extent possible 
from overlaying into adjacent locations or regions of the 
image, without actually having to identify the presence 
of specific human understandable image content in the 2 5 
adjacent regions.) In the final stage 40, adjustments are 
made to certain attributes of the item, e.g., its color, or 
to neighboring areas surrounding the item to improve 
the item's prominence in the resulting image. 

30 

Stage 1 : An election of an optima! region and an 
optimal location for the placement of the item. 

[0018] In the first stage 20, the input image is auto- 
matically analyzed to identify an optimal region and an 35 
optimal location (within the optimal region) for the place- 
ment of the item. In order to identify such a region, the 
input image is first analyzed in step 22 to identify its 
large contiguous regions that do not contain any signifi- 
cant or valuable image content. Such regions are con- 40 
sidered candidate regions for the placement of the item, 
which will typically overlay and occlude image pixels 
(that are not significant or valuable in terms of image 
content) that are in the candidate region. According to 
the preferred embodiment, there are several known 45 
methods that may be used for identifying the candidate 
regions. According to a first method, the input image is 
analyzed to detect significantly sized open spaces of 
the image, using the method and algorithm described in 
the aforementioned U.S. Patent No. 5,901,245, which is 50 
incorporated herein by reference. Such open space 
regions may be further processed to exclude some 
regions based upon characteristics such as size, shape, 
and color which are indicative of image content that 
should not be occluded or obscured or are otherwise 55 
unsuitable for item placement. For example, flesh 
colored regions, or small regions, may be excluded from 
further consideration as item locations. 



[001 9] In a second method for identifying the candi- 
date regions, the given image is analyzed to identify sig- 
nificantly sized contiguous regions not containing the 
main subject(s). This may be accomplished by employ- 
ing a main subject detection algorithm, such as the one 
described in U.S. Patent Application 09/223,860, filed 
December 31 , 1 998 in the name of the assignee of the 
present application, and which is incorporated herein by 
reference. Referring to Fig. 2, there is shown a block 
diagram of an overview of the main subject detection 
algorithm described in that patent application. First, an 
input image of a natural scene is acquired and stored in 
a step 50 in a digital form. Then, the image is seg- 
mented in a step 52 into a few regions of homogeneous 
properties. Next, the region segments are grouped into 
larger regions based on similarity measures in a step 54 
through non-purposive perceptual grouping, and further 
grouped into larger regions corresponding to perceptu- 
ally coherent objects in a step 56 through purposive 
grouping (purposive grouping concerns specific 
objects). Non-purposive perceptual grouping can elimi- 
nate over-segmentation due to large illumination differ- 
ences, for example, a table or wall with remarkable 
illumination falloff over a distance. Purposive perceptual 
grouping is generally based on smooth, noncoincidental 
connection of joints between parts of the same object, 
and in certain cases models of typical objects (for exam- 
ple, a person having head, torso and limbs). 
[0020] The regions are evaluated for their saliency 
in a step 58 using two independent yet complementary 
% types of saliency features - structural saliency features 
and semantic saliency features. The structural saliency 
features, including a set of low-level early vision fea- 
tures and a set of geometric features, are extracted in a 
step 58a, which are further processed to generate a set 
of self -saliency features and a set of relative saliency 
features. Semantic saliency features in the forms of key 
subject matters, which are likely to be part of either fore- 
ground (for example, people) or background (for exam- 
ple, sky, grass), are detected in a step 58b to provide 
semantic cues as well as scene context cues. In this 
application, the low-level early vision features include 
color, brightness, and texture. The geometric features 
include location (centrality), spatial relationship (border- 
ness, adjacency, surroundedness, and occlusion), size, 
shape, and symmetry. The semantic features include 
flesh, face, sky, grass, and other green vegetation. The 
evidences of both semantic and structural types are 
integrated in a step 60 using a reasoning engine based 
on a Bayes net to yield the final belief map of the main 
subject in a step 62. 

[0021] Both of the aforementioned methods for 
identifying the candidate regions can return a binary 
map containing the candidate regions and non-candi- 
date regions of the given image with, e.g., Vs and 0's 
assigned to the respective regions. A third method 
employs both previously identified methods and logi- 
cally combines the results (binary maps) with, e.g., a 
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Boolean operator, such as ANDing the output of the 
respective methods. In this method, additional process- 
ing of the open space map to exclude, as described 
above, some regions based upon characteristics such 
as size, shape, and color may be unnecessary. Which- 
ever method is employed, the binary map of the candi- 
date regions is hereby referred to as the candidate 
region map. 

[0022] In preparation for overlay of the desired 
item(s) into the optimal candidate region, the desired 
item(s) may need to be transformed into a format com- 
patible with the digital image in step 24. For example, 
ASCII text may need to be rendered with an appropriate 
typeface and resolution to produce a pixel map compat- 
ible with the digital image. A graphic element may need 
to be similarly rendered into a pixel map or be scaled to 
a size compatible with the digital image. This transfor- 
mation in the step 24 produces a grayscale pixel map 
representation commonly referred to as an alpha chan- 
nel. The gray value of each pixel indicates the opacity of 
the corresponding pixel in the object it represents. The 
alpha channel representation thus indicates which pix- 
els within the image object are to be overlaid into the 
digital image and how much they obscure the underly- 
ing image. 

[0023] In step 26, one of the detected candidate 
regions is selected as an optimal region for the place- 
ment of the desired item based the following criteria and 
process as set forth in Figure 3. In step 26b, the user 
may indicate a preference for the general location of the 
placement. For example, the user may specify that the 
item be placed away from the center of the image, or 
near one of its corners, etc. In doing so, a specific gray- 
scale weighting function corresponding to the specified 
location preference may be chosen by the user in a step 
26a. If the user does not express a location preference 
in step 26b, a default location preference and its associ- 
ated weighting function is chosen a step 26c by the sys- 
tem. The binary candidate region map is multiplied by 
the grayscale weighting function a step 26d producing a 
pixel suitability map. This map indicates how suitable 
each pixel is for item placement In the preferred 
embodiment, the higher the suitability value (i.e., bright- 
ness) of a pixel, the more suitable it is considered for 
item placement. 

[0024] One approach to finding the "best" candidate 
region and item location can be found by convolving the 
rendered item alpha channel representation over the 
pixel suitability map a step 26e to obtain an area suita- 
bility map. The area suitability value at a pixel in the 
area suitability map is the sum of the pixel suitability val- 
ues covered by the item when the item is placed at that 
location. The peak value(s) in the area suitability map, 
which indicate the "best" item location (s), are found in a 
step 26f. However, even with a modest size item, this 
convolution operation may become computationally 
expensive and, depending upon the processing capabil- 
ities available for the application, some shortcuts may 
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need to be taken, as follows. 

[0025] Instead of convolving the alpha channel rep- 
resentation of the item over the full size pixel suitability 
map, the pixel suitability map and the alpha channel 
5 representation are decimated a step 26g by a similar 
scale to create corresponding representations that are 
smaller and more computationally manageable. The 
convolution operation may be performed in a step 26h 
at this decreased resolution to get a coarse approxima- 
w tion of the "ideal" item location, and peak values are 
located in a step 26i. Because some of the fine details 
of the pixel suitability map and the object alpha repre- 
sentation are lost in the decimation process, this coarse 
approximation of the "ideal" location may need to be 
15 refined at full resolution. 

[0026] Once the initial position has been chosen, 
the coarse map may be discarded. This initial position is 
refined at full resolution in a step 26j in order to minimize 
the occlusion of non-candidate (non-suitable) region 
20 pixels by the item. To avoid placing the item adjacent to 
pixels of non-candidate regions, a morphologically 
dilated version of the item alpha channel is generated. 
This dilated alpha channel is placed over the initial posi- 
tion and is moved around the area of the initial position 
25 to find the best fit. This best fit search can be performed 
through convolution, or by iteratively moving the alpha 
channel representation about in a manner similar to gra- 
dient based block matching by correlation (see, e.g., 
Gonzales and Woods "Digital Image Processing" Addi- 
30 son-Wesley © 1992 Section 9.3.1 pp 583-586). In step 
26k, the user has an option to request the next best 
location or another suitable location in another preferred 
part of the image. Once a location is found, the position 
is corrected to account for the dilation differences in 
35 size, and the original item mask is placed. 

Stage 2: Acceptance or modification of the size, and 
shape of the item. 

40 [0027] In some cases, the item may have an opti- 
mal fit to a certain location in an optimal region and still 
extend beyond the optimal location or even beyond the 
optimal region itself and potentially obscure human 
understandable content in other areas of the optimal 

45 region or in regions of the digital image adjoining the 
optimal region. This may occur, for example, if the digital 
image does not have any areas of open space large 
enough for placement of the particular item. Accord- 
ingly, in the second stage 30, the item is adapted for 

so placement at the selected optimal location within the 
selected optimal region by adjustment of its size and/or 
shape. In order to adjust the size and shape of the item, 
the quality of the chosen location is measured in step 32 
by calculating the percentage of pixels in the item 

55 occluding non-candidate region pixels, which may con- 
tain human understandable image content that should 
not be obscured by an item. This percentage of 
occluded non-candidate region pixels is compared to a 
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threshold in step 34 indicative of an acceptability quality 
level. If the quality of the fit is deemed insufficient in step 
34, and the physical distribution of occluded non-candi- 
date region pixels indicates the fit may improve by trans- 
forming the size and/or shape of the image, the item 
may be transformed in step 36 in a manner that better 
suits the selected image region. For instance, the item 
may be made smaller, or if the item is text, the text may 
be broken into two or more lines. 

[0028] In the preferred embodiment, a set of prede- 
fined rules is employed to transform the item. If the more 
than 5% of a graphic item's pixels occlude non-candi- 
date region pixels and the preponderance of these 
occluded pixels occur along the periphery of the item, 
the item will be subjected to a 3% reduction in size 
maintaining the proper aspect ratio. Similarly, a text cap- 
tion may be rerendered at a decreased font size. If a text 
caption is long and contains words that by themselves 
exhibit a poor quality of fit, the caption may be broken 
into two or more lines. If the transformation results in a 
significant change in the shape of the item, such as 
breaking a caption into multiple lines, the process may 
return to step 26. 

Stage 3: Selection of color and adjustments to 
enhance prominence of the placed item. 

[0029] If the item is a colored graphic element, it 
may be overlaid into the image without changes to its 
color. Alternately, if the image consists solely of an 
alpha channel, or contains some regions within the 
graphic element for which the color may be assigned, 
the color of the item or regions of the item can be auto- 
matically chosen by the system. 

[0030] The color of the item may be optionally 
selected by the user, e.g., in the input step 10, or auto- 
matically selected in a subsequent stage. If the color of 
the item is automatically chosen, it may be chosen to 
contrast with the selected image region in which the 
item is located. The color is selected from a palette of 
colors containing a set of default colors as well as dom- 
inant colors found in the image outside the item place- 
ment area, or derived to compliment dominant colors 
found within the item placement area. The automatic 
color selection process preferably selects a color that 
results in the highest contrast with the selected region 
for item placement. The adapted and colored item is 
then placed in the selected region in step 44 at the 
selected best location. 

[0031] In the preferred embodiment color selection 
consists of the following steps. Color histograms are 
constructed of the entire image as well as the area sur- 
rounding the item placement location. Dominant colors 
represented within the whole image histogram are iden- 
tified as candidate colors. Candidate colors found to be 
dominant within the histogram of the area surrounding 
the item placement location are discarded. Candidate 
colors that do not provide adequate luminance contrast 



to the item placement area are similarly discarded. The 
remaining candidates are grouped by hue, and all but 
the most highly saturated member of each group are 
discarded. The remaining candidates are added to a list 
5 of default colors. The color providing the highest con- 
trast to the surrounding area is selected, and the 
remaining candidates are retained as options for the 
user to select. 

[0032] In the third stage 40, after the item is placed, 
io the color of the area surrounding the item is adjusted in 
a step 42 to improve the prominence of the placed item 
in the resulting image. A set of rules is used for this pur- 
pose. For example, if a significant number of pixels 
within an area around the item are of matching or similar 
75 colors to the color of the item, then the pixels within an 
area around the item may be darkened or lightened to 
force an improvement in contrast or blend. Similarly, the 
area surrounding the item may be darkened or light- 
ened if the placement region contains too many pixels 
20 belonging to non-candidate regions. 

[0033] The image as finally rendered with the item 
overlaid into the selected open space region is pre- 
sented to the user via the display (see Figure 3) for 
approval in step 44. If the user indicates approval in step 
25 46 then the job is done; otherwise the process is looped 
back through a user adjustment step 48 until the item is 
rendered and overlaid in a manner acceptable to the 
user/viewer. Two possibilities are illustrated in Figure 1. 
If the color of the item is unacceptable, the user may 
3o make another color selection in the step 48 and the 
process returns to step 44 - in effect, the user can be 
presented with a palette of colors and step through 
them one by one until an acceptable color is found. If the 
size and/or shape, or the selected location for the over- 
35 lay, is unacceptable, then the process is returned to step 
24 where the item is again transformed in preparation 
for overlay. In this situation, the user may desire to pro- 
vide certain overriding considerations, such as size or 
typeface as well as a predetermined open space region, 
40 in the user adjustment step 48. The process is then 
repeated until a satisfactory result is obtained. 
[0034] The subject matter of the present invention 
is related to digital image understanding technology. 
[0035] Other aspects of the invention include: 

45 

1. A method for placing a human understandable 
item in a digital image so as enhance the promi- 
nence of the item in relation to human understand- 
able content in the digital image, said method 
so comprising the steps of: 

(a) digitally processing the digital image to rec- 
ognize and identify an optimal location in the 
digital image for placing the item; and 
55 (b) modifying one or more attributes of the item 

in relation to the human understandable image 
content to enhance the prominence of the item 
in the digital image. 
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2. A method in which the step of digitally processing 
the digital image to recognize and identify an opti- 
mal location in the digital image for placing the item 
comprises identifying an optimal region comprising 
open space in the digital image and then finding an 5 
optimal location within the optimal region. 

3. A method in which the step of modifying one or 
more attributes of the item in relation to the human 
understandable image content to enhance the io 
prominence of the item in the digital image com- 
prises modifying one or more attributes of the item 

in relation to the human understandable image con- 
tent in the digital image outside of the optimal loca- 
tion. 75 

4. A method in which the step of modifying one or 
more attributes of the item in relation to the human 
understandable image content to enhance the 
prominence of the item in the digital image com- 20 
prises modifying at least one of the color, size and 
shape of the item to contrast with the human under- 
standable image content. 

5. A computer program product for placing a human 
understandable item in a digital image comprising: 
a computer readable storage medium having a 
computer program stored thereon for performing 
the steps of: 

(a) digitally processing the digital image to rec- 
ognize and identify an optimal location in the 
digital image for placing the item; and 

(b) modifying the placement of the item in rela- 
tion to human understandable image content 
so as to minimally obscure such human under- 
standable image content in the digital image. 

6. A computer program product in which the step of 
digitally processing the digital image to recognize 
and identify an optimal location in the digital image 
for placing the item comprises identifying an opti- 
mal region comprising open space in the digital 
image and then finding an optimal location within 
the optimal region. 

7. A computer program product in which the step of 
modifying the placement of the item in relation to 
human understandable image content so as to min- 
imally obscure such human understandable image 50 
content in the digital image comprises modifying 

the placement of the item in relation to human 
understandable image content so as to minimally 
obscure such human understandable image con- 
tent in other locations of the optimal region. 55 

8. A computer program product in which the step of 
modifying the placement of the item in relation to 
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human understandable image content so as to min- 
imally obscure such human understandable image 
content in the digital image comprises modifying 
the placement of the item in relation to human 
understandable image content so as to minimally 
obscure such human understandable image con- 
tent in a region of the digital image adjoining the 
optimal region. 

9. A computer program product in which the step of 
modifying the placement of the item in relation to 
human understandable image content so as to min- 
imally obscure such human understandable image 
content in the digital image comprises evaluating 
the proportion of image pixels in the item that 
occlude pixels outside of either the optimal location 
or the optimal region. 

10. A computer program product further comprising 
a step of modifying one or more attributes of the 
item in relation to the human understandable image 
content in the digital image to enhance the promi- 
nence of the item in the digital image. 

1 1 . A computer program product in which the step 
of modifying one or more attributes of the item in 
relation to the human understandable image con- 
tent in the digital image to enhance the prominence 
of the item in the digital image comprises modifying 
one or more attributes of the item in relation to the 
human understandable image content in the digital 
image outside of the optimal location. 

12. A computer program product in which the step 
of altering one or more attributes of the item com- 
prises altering at least one of the color, size and 
shape of the item. 

13. A method for automatically placing text or a fig- 
urative element within an image so as to minimize 
the amount of visual image information obscured, 
the method comprising the steps of: 

(a) examining the image to determine a suita- 
ble area for placement of the text or figurative 
element; 

(b) determining an optimal location within the 
suitable area for the text or figurative element to 
be placed; 

(c) determining a goodness of fit within the opti- 
mal location for the text or figurative element; 

(d) transforming the text or figurative element 
based upon the results of step (c) to improve 
the goodness of fit for the text or figurative ele- 
ment; 

and 

(e) determining an optimal color for the text or 
figurative element based upon the location of 



: <EP 1085464A2_I_> 



13 



EP 1 085 464 A2 



• 



the text or figurative element in the image. 

14. A method wherein the step of examining the 
image to determine a suitable area for placement of 
the text or figurative element further includes find- 
ing regions of the image devoid of significant image 
content. 

15. A method wherein the step of examining the 
image to determine a suitable area for placement of 
the text or figurative element further includes find- 
ing open spaces within the image. 

16. A method wherein the step of examining the 
image to determine a suitable area for placement of 
the text or figurative element further includes find- 
ing regions of the image not containing the main 
subject. 

17. A method wherein the step of determining an 
optimal location within the suitable area of the text 
or figurative element to be placed further includes 
searching the image for the location where the text 
or figurative element occludes a minimum of signif- 
icant visual image content. 

18. A method wherein the step of transforming the 
text or figurative element based upon the results of 
the step of determining a goodness of fit within the 
optimal location for the text or figurative element to 
improve the goodness of fit for the text or figurative 
element further includes scaling the text or figura- 
tive element. 

19. A method wherein the step of transforming the 
text or figurative element based upon the results of 
the step of determining a goodness of fit within the 
optimal location for the text or figurative element 
further includes splitting the text into multiple lines 
with various justification. 

20. A method wherein the step of determining an 
optimal color for the text or figurative element based 
upon the location of the text or figurative element in 
the image further includes choosing white or black 
as the color of the text so as to maximize the con- 
trast between the text and the surrounding image 
area. 

21 . A method wherein the step of determining an 
optimal color for the text or figurative element based 
upon the location of the text or figurative element in 
the image further includes choosing a dominant 
color appearing within the image as the color of the 
text so as to maximize the contrast between the text 
and the surrounding image area. 

22. A method wherein the step of determining an 



optimal color for the text or figurative element based 
upon the location of the text or figurative element in 
the image further includes choosing a color of the 
text so as to maximize the contrast between the text 
and the surrounding image area. 
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A method for placing a human understandable item 
in a digital image, said method comprising the steps 
of: 

(a) digitally processing the digital image to rec- 
ognize and identify an optimal location in the 
digital image for placing the item; and 

(b) modifying the placement of the item in rela- 
tion to human understandable image content 
so as to minimally obscure such human under- 
standable image content in the digital image. 



2. A method as claimed in claim 1 in which step (a) 
comprises identifying an optimal region comprising 
open space in the digital image and then finding an 
optimal location within the optimal region. 

3. A method as claimed in claim 2 in which step (b) 
comprises modifying the placement of the item in 
relation to human understandable image content so 
as to minimally obscure such human understanda- 
ble image content in other locations of the optimal 
region. 

4. A method as claimed in claim 1 in which step (b) 
comprises modifying the placement of the item in 
relation to human understandable image content so 
as to minimally obscure such human understanda- 
ble image content in a region of the digital image 
adjoining the optimal region. 

5. A method as claimed in claim 1 in which step (b) 
comprises evaluating the proportion of image pixels 
in the item that occlude pixels outside of either the 
optimal location or the optimal region. 

45 6. A method as claimed in claim 1 in which the items 
include at least one of a caption, text and a figura- 
tive element. 

7. A method as claimed in claim 1 further comprising 
so a step (c) of modifying one or more attributes of the 
item in relation to the human understandable image 
content in the digital image to enhance the promi- 
nence of the item in the digital image. 

55 8. A method as claimed in claim 7 in which step (c) 
comprises modifying one or more attributes of the 
item in relation to the human understandable image 
content in the digital image outside of the optimal 
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location. 

A method as claimed in claim 7 in which the step (c) 
of altering one or more attributes of the item com- 
prises altering at least one of the color, size and 5 
shape of the item. 
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